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Along  with  our  efforts  to  prepare  an  operational  prototype  for  a  wide  area  surveillance  system,  identity 
fusion  was  considered  as  one  of  the  major  system  components.  Many  problems  have  been  detected  about 
how  the  sensors  produce  the  identity  information,  how  they  sent  this  information,  registration  of  the 
information,  detecting  the  conflicts,  combining  the  appropriate  information  in  an  effective  way, 
the  presentation  of  the  final  fused  information  to  the  operators,  and  consequently  interpreting  the  results 
for  the  coordination  of  sensors  for  further  identification  efforts.  On  the  other  hand,  the  appropriate  way  of 
integrating  identity  information  to  the  rest  of  the  fusion  process  constitutes  the  other  important  context.  In 
this  paper,  we  report  our  experience  for  a  wide  area  surveillance  system  from  identification  fusion 
perspective  based  on  the  prototype  system  developed  in  Turkish  Navy  Research  Center  Command. 


1.0  INTRODUCTION 

Different  type  of  threats,  changing  hostile  activities,  dense  operational  areas,  littoral  waters,  and 
asymmetric  threat  expectations  increase  the  significance  of  the  classification  and  identification  process  in 
a  maritime  surveillance  system.  Although  many  different  types  of  evolving  sensors  are  utilized  in  modem 
surveillance  systems,  it  is  still  a  challenging  task  to  establish  and  maintain  the  “recognized  tactical 
picture”. 

As  the  threats  and  threat  evaluations  have  been  changing,  associated  identification  process  has  become 
more  complex  in  military  surveillance  systems.  Littoral  Warfare  and  operations  other  than  war  concepts 
has  created  different  and  more  challenging  requirements  in  both  the  content  and  timing  requirements  of 
the  identification  information  of  the  tactical  picture. 

As  the  requirements  are  getting  more  challenging,  the  surveillance  systems  are  getting  better  especially  by 
adapting  their  sensors  to  a  “network  centric”,  “network  oriented”  or  “network  enabled”  systems. 
But  unfortunately,  despite  very  nice  definitions  and  explanations,  we  observe  that  there  are  still  some  very 
important  technical  challenges  to  achieve  in  network  centric  applications. 

Different  multi  sensor  data  fusion  techniques  are  utilized  in  establishing  the  recognized  maritime  tactical 
picture.  In  most  general  sense,  the  first  level  of  multi-sensor  data  fusion,  known  as  parametric  fusion  is 
considered  in  two  main  stages:  Positional  fusion  and  identity  fusion.  On  the  other  hand,  the  interactions  of 
the  second  and  third  level  fusion,  namely  situation  assessment  and  threat  evaluation  is  not  always  a  clear 
cut  especially  when  the  identification  fusion  is  considered. 

In  this  paper,  we  cover  some  of  the  problems  that  we  faced  in  a  prototype  maritime  surveillance  system 
from  identification  fusion  perspective.  To  cover  the  material  in  an  organized  way,  we  group  the  problems 
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as  technical  and  operational  problems;  where  as  technical  problems  can  further  be  grouped  as  architectural 
and  fusion  process  aspects. 

We  introduce  the  problem  in  more  detail  with  a  brief  explanation  of  our  prototype  system  in  section  2. 
In  section  3  we  cover  the  technical  problems  covering  both  architecture  and  fusion  process  aspects. 
Section  4  deals  with  operational  problems.  Section  5  concludes  our  report. 


2.0  PROBLEM  DEFINITION 

A  wide-area  surveillance  system  is  a  good  example  of  multi-sensor,  multi-site  fusion  system  with 
potentially  many  different  type  and  number  of  systems,  located  in  different  geographical  locations  and 
covering  many  different  types  of  “targets”. 

Such  a  surveillance  system  has  been  prototyped  in  Turkish  Navy  Research  Center  Command  to  try  out 
various  techniques  in  actual  system  development.  Main  components  of  the  prototype  system  are: 

•  Distributed  sensor  simulation  system 

•  Fusion  node  prototype 

•  Sensor  Integration  Units 

•  Fusion  and  Command  Tactical  System  (FACTS) 

•  Communication  media  simulation  system 

In  the  distributed  sensor  simulation  environment  you  can  create  off-line  scenarios.  These  scenarios  can  be 
modified  during  the  run-time  as  they  provide  the  ground-truth  information  for  the  sensor  simulators.  When 
these  scenarios  are  run,  sensor  simulators  receives  the  ground-truth  information,  evaluate  their  coverage, 
and  creates  sensor  tracks  with  a  medium  fidelity,  which  we  consider  as  satisfactory  for  our  prototype 
purposes. 

After  the  appropriate  track  management,  the  sensors  report  their  track  information  to  the  fusion  center,  via 
very  realistic  communication  media  simulation  system,  which  is  in  our  case  a  land-based  communication 
system. 

In  the  prototype  fusion  node,  where  all  of  the  information  is  collected  and  processed,  FACTS,  has  the 
main  processing  functionalities.  It  has  mainly  two  different  modules: 

•  Multi-sensor  data  fusion  module  (MSDF),  and 

•  Tactical  level  fusion  module  (TM) 

where  MSDF  module  performs  actual  fusion  process  and  TM  provides  services  for  operator  input  and 
presentations  for  higher  levels  evaluations  and  interactions. 

At  this  point  we  need  to  emphasize  the  differences  between  the  organic  and  non-organic  sensors  in  such  a 
fusion  system.  Sensors  which  are  controlled,  operated  and  monitored  directly  within  the  fusion  system  are 
considered  as  organic  sensors,  where  as  sensors  or  systems  which  provide  information  to  the  fusion 
system  indirectly,  mostly  after  an  evaluation  process  are  considered  as  non-organic  sensors. 

Within  this  context,  MSDF  has  been  designed  and  implemented  in  order  to  fuse  data  and  information  from 
different  number  and  types  of  organic  sources.  Modem  fusion  algorithms  and  techniques  can  be  studied 
within  this  module.  On  the  other  side,  TM  is  designed  to  integrate  non-organic  data  and  operator 
evaluations  into  the  system. 
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In  a  wide-area  maritime  surveillance  system  that  can  be  simulated  effectively  with  the  above  described 
prototype  system,  we  faced  many  problems  starting  from  very  basic  concepts  regarding  the  simulation 
process  up  to  the  distribution  of  the  recognized  maritime  tactical  picture  established.  We  choose 
evolutionary  prototyping  method.  It  was  useful  since  it  has  provided  a  common  environment  to  exchange 
ideas  and  evaluate  the  results  before  the  system  gets  unmanageable.  Another  important  system  wide 
problem  was  to  decide  about  the  alternative  algorithms  to  be  implemented  in  the  prototype  system. 
Different  risk  evaluations,  concerning  performance,  infrastructure,  integration,  and  available  resources  has 
led  sometimes  the  selection  of  not  the  best  algorithm  evaluated. 

Two  main  important  modules  of  parametric  fusion,  positional  and  identity  fusion  are  designed  and 
implemented  in  an  integrated  but  modular  manner.  Although  there  are  some  recent  efforts  in  the  literature 
to  integrate  the  identity  information  in  the  early  stages  of  the  positional  fusion,  even  in  the  association 
phase,  to  get  better  association  results,  we  have  utilized  the  methods  which  integrate  the  identity 
information  in  later  stages  of  the  fusion  process,  especially  after  the  positional  fusion  process. 

In  order  to  explain  the  problems  from  identification  process,  which  we  concentrate  in  this  paper,  we  divide 
these  problems  in  two  different  groups  as  technical  and  operational.  Technical  problems  are  further 
detailed  as  architectural  and  identification  fusion  process  related. 


3.0  TECHNICAL  PROBLEMS 

There  are  some  technical  challenges  to  be  taken  into  account  in  any  large-scale  fusion  system.  These 
problems,  which  we  named  as  architectural  problems  are  listed  below  for  the  sake  of  completeness  and 
detailed  in  [2],  These  architectural  problems  are: 

•  Different  types  of  sensors, 

•  Different  types  of  systems,  subsystems, 

•  Different  types  of  communication  media  and  communication  requirements, 

•  Different  types  of  information  processing  techniques, 

•  Effective  presentation  to  the  operators  and  to  the  command  team,  (Human  Machine  Interface), 

•  Effective  sensor  and  resource  management. 

On  the  other  hand,  there  are  some  problems  that  we  faced  related  to  identification  process  specifically. 
These  problems  are  defined  in  [3]  to  some  extend.  We  enhance  these  explanations  based  on  our  experience 
in  the  rest  of  this  section. 

First  of  all,  due  to  its  significance  we  emphasize  that  sensor  related  issues  constitutes  a  great  problem  by 
itself,  due  to: 

•  Different  types  of  sensors  reporting  different  types  of  identification  information, 

•  Unknown  characteristics  of  the  sensors, 

•  Uncertainties  associated  with  their  reports, 

•  Difficulties  to  enforce  the  sensor  manufacturers  to  think  appropriately  for  fusion  purposes. 

Most  of  the  sensors  are  generally  designed  in  more  of  a  stand-alone  system.  As  the  integrated  systems, 
systems  of  systems  and  finally  network-centric  warfare  concepts  evolves,  it  becomes  inevitable  to  require 
some  changes  in  the  specification  of  the  sensors.  Acknowledging  the  increasing  significant  role  of  the 
“system  engineers”  role  in  large-scale  projects,  system  engineers  have  to  consider  many  different  topics, 
which  will  definitely  include  the  very  detailed  capabilities  of  each  sensor  providing  some  kind  of  data  to 
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the  fusion  process.  In  many  cases,  knowing  only  the  capabilities  of  the  sensors  will  not  be  enough,  but  the 
algorithms  and  even  some  implementation  details  of  the  sensors  will  be  necessary  for  the  success  of  the 
overall  system  performance. 

As  the  basic  step  of  the  parametric  fusion  process,  association  (or  in  some  literature  correlation)  serves 
both  for  the  positional  and  identification  fusion  purposes.  A  successful  association  mechanism  is  the  key 
element  for  a  successful  identification  process.  Assuming  that  association  has  been  done  successfully,  we 
can  enhance  identification  fusion  problems  as  follows: 

•  Processing  combat  id  (hostile,  friend,  suspect,  etc)  and  identification  information  (type  A,  Class 
B,  platform  X)  separately  is  a  major  design  decision  in  the  system.  It  is  partially  due  to  the  fact 
that  the  significance  of  these  two  different  levels  of  classification  has  different  operational 
requirements.  In  any  case,  the  first  goal  is  to  provide  a  combat  id  to  the  operator.  Combat  id  is 
more  fundamental  and  it  has  some  well-known  regulations  and  rules  defined  in  operational 
documents.  On  the  other  hand,  identification  information  provides  more  information  about  a 
specific  target.  Although  there  is  a  close  relationship  between  combat  id  and  identification 
information,  these  two  can  be  processed  independently  up  to  a  certain  decision  level. 
Identification  information  after  having  some  conflict  checks  with  the  combat  id,  will  be  used  for 
situation  assessment  and  threat  evaluation  purposes. 

•  Concentrating  on  the  identification  information  and  its  fusion  process,  we  need  to  consider 
different  identification  information  produced  by  the  sensors.  For  most  of  the  sensors,  it  is  not  very 
well  defined  how  they  produce  the  identification  information  associated  with  a  specific  track. 
Hence  very  careful  study  of  the  sensor  data  is  required.  Most  of  the  time,  understanding  of  the 
internal  sensor  processing  is  vital  for  a  successful  fusion  process. 

•  When  these  data  or  information  is  sent  to  Fusion  center  or  fusion  node,  appropriate  registration 
mechanisms  are  to  be  done  effectively.  This  covers  converting  possibly  different  types  of 
information,  such  as  possibility,  probabilistic,  linguistic  and  likelihood  values  to  a  common 
information  domain. 

•  Detecting  the  possible  conflicting  information  reported  by  the  same  sensor  and  resolving  these 
conflicts,  either  automatically  or  via  interactions  with  the  operator  constitutes  the  next  step  in  the 
fusion  process.  Conflicting  reports  from  a  specific  sensor  may  affect  the  result  of  the 
identification  fusion  process  significantly,  hence  we  think  that  before  making  such  reports 
effective  on  the  fusion  process,  potential  conflicts  have  to  be  checked  and  resolved.  Actually  it  is 
a  way  of  having  a  safety-point  in  the  identification  fusion  process,  since  we  are  not  sure  about  the 
quality  of  the  identification  data/information  sent  by  the  sensors  automatically.  In  order  to  be  able 
to  keep  the  system  reliable  for  the  end  user,  we  think  that  it  is  worthwhile  to  put  some  more  effort 
(computer  and  operator  if  needed)  to  check  for  the  inconsistent  reports. 

•  Similar  conflict  checks  are  also  to  be  performed  within  the  reports  of  different  sensors  for  a 
specific  target.  Detecting  the  possible  conflicting  information  reported  by  different  sensors  and 
resolving  these  conflicts  is  necessary  for  maintaining  reliable  results. 

•  After  these  conflict  checks  identification  information  combining  process  takes  place.  Bayesian 
and  Dempster-Shafer  models  form  two  main  approaches  for  combining  the  identification 
information.  We  have  also  got  some  experience  with  the  Transferable  Belief  Model  (TBM)  [4], 
which  is  actually  an  interpretation  of  Dempster-Shafer  model.  Although  we  have  not  a  clear-cut 
idea  on  which  algorithm  to  utilize  for  the  final  system,  TBM  gives  promising  results  for  scenarios 
with  specific  problems. 

•  The  presentation  of  the  combined  information  to  the  operator  in  an  understandable  format  is  still 
an  important  issue.  The  operator  needs  to  have  access  to  the  information  sent  by  the  sensors, 
the  identification  information  calculated  by  the  system  (if  necessary  after  a  pruning  process)  and 
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some  resultant  identification  information  that  is  either  generated  by  the  system  or  operator. 
Operator  needs  to  be  able  to  modify  the  resultant  identification  information  even  if  it  is  generated 
by  the  system  automatically. 

•  On  the  other  hand  the  content  of  the  identification  information  presented  to  the  operator  is  also 
important.  Especially  in  DS  and  TBM  methods,  basic  probability  assignments  (mass  values), 
Belief  and  Plausibility  values  calculated  after  the  combining  process,  are  difficult  to  be  visualized 
by  the  operators,  especially  by  the  end  users.  Hence  either  a  very  detailed  training  shall  be  given 
to  the  operators,  or  automatic  conversions  have  to  be  implemented  for  more  custom  values  such  as 
probability  values.  A  conversion  method  is  proposed  in  [4]  by  Delmotte  and  Smets. 

•  An  additional  consistency  check  is  very  useful  between  the  resultant  identification  information 
and  the  other  data  gathered  from  different  sources  such  as  intelligent  reports,  operational 
messages  and  even  information  via  a  Geographical  Information  system. 

•  In  almost  every  step  there  is  a  conflicting  check  requirement.  This  is  mostly  due  to  the 
requirement  to  maintain  the  high  level  of  reliability  through  out  the  system.  In  most  cases,  having 
no  result  is  considered  as  better  than  producing  a  wrong  result.  Hence  we  prefer  to  process  the 
data  automatically  as  much  as  we  can,  but  if  there  is  a  doubt  the  task  is  to  alert  the  operator  about 
the  problematic  case. 

•  The  effect  of  the  identification  information  fusion  on  the  positional  fusion  results,  especially  on 
the  association  process  has  also  to  be  taken  care  of  by  the  system.  Especially  a  conflict  report  is 
approved  by  the  operator  for  a  track  reported  by  more  than  one  sensor  then  operator  shall  be 
provided  with  the  tools/services  to  maintain  the  association  relation  appropriately. 

•  As  it  can  be  followed  all  the  issues  that  we  mentioned  are  strongly  related  to  the  organic  sensor 
information  process.  There  are  also  some  additional  problems  in  the  Tactical  Level  Fusion  process 
from  the  technical  challenges  of  the  identification  fusion  perspective: 

•  The  management  of  identification  information  retrieved  from  more  than  one  link  shall  be 
handled  appropriately. 

•  For  the  conflict  resolution  between  different  links  and  organic  sensors,  we  utilize  operator 
support.  But  automation  of  identification  information  assignment  can  be  performed  to  a 
certain  extend  without  any  operator  interaction. 


4.0  OPERATIONAL  PROBLEMS 

While  there  are  some  technical  challenges  in  the  identification  fusion  process,  operational  requirements 
constitute  the  other  important  aspect  of  the  identification  fusion  process.  Our  explanation  will  start  from 
the  sensors,  continue  with  the  communication  media  and  conclude  with  the  fusion  node  itself. 

The  independent  sensors  and  their  operators  must  have  a  clear  idea  about  the  general  overall  fusion 
concept.  They  may  even  be  warned  against  the  results  of  their  ignorance  in  their  scope.  On  the  other  hand, 
this  may  also  affect  the  track  exchange  policies  of  the  fleet,  which  may  cause  some  extra  difficulties  to 
maintain  the  existing  link  communication.  Nevertheless,  the  important  thing  is  that  a  track  that  is  not 
important  for  one  sensor  may  be  vital  for  another  unit.  Hence  we  need  to  exchange  the  track  information 
to  the  extend  possible  without  any  filtering  if  possible.  Since  this  will  most  likely  cause  a  bottleneck  in  the 
communication  media,  alternative  communication  possibilities  or  operational  procedures  shall  be  utilized. 

The  significance  of  the  communication  capabilities  can  be  seen  very  easily  in  such  an  integrated 
surveillance  system.  Most  of  the  time,  alternative  communication  mechanisms  are  to  be  established  both 
to  collect  data  and  to  distribute  the  resultant  tactical  picture  effectively. 
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Secure  voice  capabilities  between  appropriate  commanders  and  secure  coordination  networks  for  specific 
purposes,  such  as  EW  or  AAW  is  of  special  interest.  Again  due  to  the  limitation  of  the  available 
communication  channels,  some  of  the  coordination  may  be  supported  via  text  messaging  between  the 
operators  or  even  between  the  commanders. 

As  we  mentioned  briefly,  assigning  an  appropriate  combat  id  for  a  track  is  more  important  than  producing 
an  identification  information  for  that  specific  track.  But  in  any  case,  they  need  to  be  complementary. 
Any  conflicting  case  is  a  potential  issue  to  investigate  for  the  tactical  picture  compilers.  This  process  can 
easily  be  automated  to  save  time  and  man  power. 

Commander/  Decision  makers  shall  not  be  expected  to  know  all  the  details  of  the  surveillance  and 
identification  fusion  process,  but  instead  there  shall  be  some  other  people  who  will  be  in  charge  of 
coordination  of  the  surveillance  process  and  the  evaluation  of  the  intermediate  results.  In  most  cases,  it  is  a 
good  idea  to  have  one  or  more  identification  officer  to  support  the  commander  for  these  tasks. 

Identification  officer  should  have  detailed  information  about  the  capabilities,  limitations,  and  even  the 
underlying  algorithms  of  the  integrated  sensors  and  communication  systems  of  the  overall  surveillance 
system. 

Operational  organization  of  the  fusion  node  or  in  more  general  sense  fusion  center  is  very  important  to 
achieve  the  targeted  goal.  Although  the  organization  is  to  be  flexible  with  respect  to  the  given  scenario,  the 
workflow  of  the  fusion  center  is  very  important  for  the  successful  result.  Within  this  context,  the 
presentation  of  the  tactical  picture  internally,  the  filtering  mechanisms,  switching  between  different 
operator  console  screens,  the  video-wall  applications,  internal  communication  capabilities  are  all  to  be 
considered  as  parts  of  workflow  of  the  fusion  center. 

It  is  our  experience  that  to  enable  some  operators  to  access  the  detailed  information  gives  a  good 
confidence  over  the  system.  Hence  the  related  operators,  including  the  identification  operator,  shall  have 
access  to  the  sensor  reports,  their  details  and  possibly  the  history  of  the  sensor  reports;  the  calculated 
identification  information  produced  by  the  sensor,  the  pruned  version  of  the  calculated  identification 
information  if  necessary  and  the  resultant  identification  information. 

In  order  to  establish  and  maintain  the  recognized  tactical  picture  appropriately,  the  coordination  of  the 
sensors  and  available  resources  shall  also  be  considered  as  an  important  operational  aspect.  Especially  the 
allocation  of  extra  resources  to  identify  a  specific  track  requires  a  comparison  of  potential  threat  and 
necessary  resource  allocation.  We  think  that  there  will  be  some  automated  tools  to  support  the  decision 
makers  for  this  specific  resource  allocation  problem. 

The  presentation  of  the  compiled  picture  to  the  operator  and  command  team  is  still  another  problematic 
issue.  There  are  no  generally  accepted  standards  to  present  the  tactical  picture  so  that  operators  and 
command  team  can  evaluate  the  current  situation  effectively.  On  the  other  side,  in  terms  of  human 
machine  interface  (HMI)  presentation  is  not  the  only  issue.  Our  concern  is  that  the  reliability  and 
confidence  of  the  overall  system  as  observed  and  evaluated  by  its  operators  and  users  is  much  more 
important  and  vital.  We  have  observed  many  systems  not  being  used  effectively  just  because  of  the 
missing  feelings  of  the  confidence.  Hence,  great  care  shall  be  given,  not  to  disturb  the  confidence  of  the 
operator  to  the  system,  as  a  continuous  process  starting  from  analysis  of  the  system  up  to  and  including 
maintenance  phase  of  the  system. 


5.0  RESULTS 

Our  experience  in  the  prototype  system  showed  that  sensors,  their  characteristics,  and  (sometimes) 
unreleased  parts  have  significant  effect  in  selecting,  designing  and  implementing  the  appropriate  fusion 
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system.  As  there  is  no  well-defined  procedure  for  identification  fusion  process,  we  benefit  our  modular 
architecture  so  that  we  can  plug  and  test  any  kind  of  identification  fusion  algorithms  in  our  system  or  we 
can  enhance  our  existing  implementation.  One  of  our  conclusions  is  that  belief  theory  deserves  more 
attention  in  the  identification  fusion  implementations  with  its  superior  characteristics  fitting  our  purposes. 
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